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ABSTRACT 

We have exploited the large area coverage of the combined UKIDSS Ultra Deep Survey 
(UDS) and Subaru/XMM-Newton Deep Survey (SXDS) to search for bright Lyman-break 
galaxies (LBGs) at z > 5. Using the available optical+near-infrared photometry to efficiently 
exclude low-redshift contaminants, we identify nine z > 5 LBG candidates brighter than 
z'ab = 25 within the 0.6 square degree overlap region between the UDS early data release 
(EDR) and the optical coverage of the SXDS. Accounting for selection incompleteness, we es- 
timate the corresponding surface density of z > 5 LBGs with z'ab < 25 to be 0.005 ± 0.002 
per square arcmin. Modelling of the optical+near-infrared photometry constrains the candi- 
dates' redshifts to lie in the range 5.1 < z < 5.9, and provides estimates for their stellar 
masses. Although the stellar mass estimates are individually uncertain, a stacking analysis 
suggests that the typical stellar mass of the LBG candidates is > 5 x 10 10 M Q which, if con- 
firmed, places them amongst the most massive galaxies currently known at z > 5. It is found 
that ACDM structure formation can produce sufficient numbers of dark matter halos at z > 5 
to accommodate our estimated number density of massive LBGs for plausible values of erg 
and the ratio of stellar to dark matter. Moreover, it is found that recent galaxy formation mod- 
els can also account for the existence of such massive galaxies at z > 5. Finally, no evidence 
is found for the existence of LBGs with stellar masses in excess of ~ 3 x 10 11 M Q at this 
epoch, despite the large co-moving volume surveyed. 

Key words: galaxies: high-redshift - galaxies: evolution - galaxies: formation 



1 INTRODUCTION 

The increasing availability of survey fields with deep multi- 
wavelength data has led to so-called "dropout" techniques becom- 
ing established for photometrically identifying high-redshift galax- 
ies. At redshifts of z ~ 6 it is now standard practice to select galaxy 
candidates using the i— drop technique, a straightforward extension 
of the Lyman-break method pioneered at z ~ 3 by Guhathakurta, 
Tyson & Majewski (1990) and Steidel & Hamilton (1992). Within 
this context, the unique combination of depth and image qual- 
ity provided by the Hubble Space Telescope (HST) has arguably 



made the largest contribution to our understanding of high-redshift 
galaxies. Analysis of the Hubble deep fields, the GOODS fields, 
and in particular the Hubble ultra-deep field (HUDF), has led to 
the identification of hundreds of i— drop galaxy candidates at faint 
(•£850 > 26 1 ) magnitudes (e.g. Bouwens et al. 2006; Malhotra et 
al. 2005; Bunker et al. 2004; Dickinson et al. 2004). As a conse- 
quence, we now have a vastly improved understanding of the galax- 
ies which undoubtedly dominate the star-formation density of the 
Universe during the epoch immediately following reionisation. 

However, although HST has greatly advanced our knowledge 
of high-redshift galaxies, the various HST deep fields suffer from 
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one fundamental weakness. Crucially, their small area (e.g. the 
HUDF covers ~13 square arcmin) makes them subject to signif- 
icant cosmic variance (Somerville et al. 2004) and completely un- 
suitable for investigating the most luminous, rarest objects. Indeed, 
of the many hundreds of i— drop galaxies identified in the deep 
HST fields, only one has zs^o < 25 (SBM03#3, spectroscopically 
confirmed at z = 5.78; Bunker et al. 2003). This is an important 
shortcoming because accurately determining the number density of 
massive galaxies at high-redshift (z > 5) has the potential to place 
important new constraints on galaxy formation models, only 1 Gyr 
after the Big Bang. From an observational perspective, it is clear 
that what is required is a combination of both depth and large area 
coverage. 

In this respect, the study of z ~ 6 LBGs in the Subaru Deep 
Field (SDF; Maihara et al. 2001) by Shimasaku et al. (2005) is per- 
haps the most notable. Based on deep Subaru optical imaging cov- 
ering 767 square arcmin, Shimasaku et al. identified twelve bright 
LBGs with z'-band magnitudes in the range 25.4<z'<26.6, pho- 
tometrically selected to lie in the redshift range 5.6 < z < 6.2. 
Moreover, by using two medium-band filters (zb and zr) which 
divide the Subaru z'-prime filter in half, Shimasaku et al. gained 
crucial information on the UV slope of their high-redshift candi- 
dates. As a consequence, Shimasaku et al. were able to select a very 
clean sample, which should have negligible contamination from 
low-redshift interlopers. In addition, Ota et al. (2005) performed 
a search for z'< 26 LBGs over a sky area of ~ 1 square degree in 
the SXDS field, using a primary selection criterion of i' — z > 1.5. 
However, Ota et al. were only able to strongly constrain the surface 
density of LBG candidates at z' > 25, because at brighter magni- 
tudes their lack of near-infrared data made it impossible to exclude 
significant contamination from low-redshift interlopers, and cool 
galactic stars in particular. Finally, it should be noted that there is 
already evidence for a large population of z ~ 6 galaxies at fainter 
z 1 — band magnitudes within the SXDS/UDS field. Using a combi- 
nation of the broad i! — band SXDS imaging and a narrow band fil- 
ter (A c = 8150A), Ouchi et al. (2005) identified 515 potential Lya 
emitters (LAEs) within the redshift interval 5.65 < z < 5.75, over 
an area of one square degree. Further deep optical spectroscopy 
subsequently confirmed eight LAEs at z ~ 5.7 with asymmetric 
Lya emission lines. However, the LAEs detected by Ouchi et al. 
are fainter (z' > 25) than the objects identified in this study (and 
presumably have lower mass). 

In this paper we exploit two crucial advantages of the com- 
bined SXDS/UDS data-set to explore a new area of LBG parameter 
space at z > 5. Firstly, the overlap region between the SXDS and 
the UDS EDR is > 2.5 times larger than the SDF (~ 0.6 square de- 
grees), offering the prospect of identifying the rarest z > 5 LBGs 
with z' < 25. Secondly, the near-infrared data from the UDS EDR 
allows us to effectively clean the sample of low-redshift contami- 
nants, and to obtain stellar mass estimates for the LBG candidates. 
The structure of the paper is as follows. In Section 2 we give a brief 
review of the optical and near-infrared data currently available in 
the SXDS/UDS field. In Section 3 we outline the criteria adopted 
for selecting the 5 < z < 6 galaxy candidates, and discuss the pos- 
sible sources of contamination. In Section 4 we present the results 
of our modelling of the candidates' spectral energy distributions 
(SEDs). In Section 5 we estimate the number densities of massive 
LBGs at z > 5 and compare with the predictions of current galaxy 
formation models. In Section 6 we compare our results with those 
in the recent literature, before presenting our conclusions in Sec- 
tion 7. Throughout the paper we adopt the following cosmology: 
H = 70 km s _1 Mpc _1 , Q m = 0.3, ft A = 0.7. When referring 



to the i and z— band filters, i' and z specifically refer to the Subaru 
filters, whereas isn and zsso are the equivalent HST filters. 



2 THE DATA 

In this paper we identify potential massive galaxies at high redshift 
by combining optical data from the Subaru telescope with near- 
infrared data from the UK Infrared Telescope (UKIRT). The optical 
data were taken as part of the Subaru/XMM-Newton Deep Survey 
(Sekiguchi et al. 2005) while the near-infrared data has been taken 
from the early-data release (Dye et al. 2006) of the UKIRT Infrared 
Deep Sky Survey (Lawrence et al. 2006). The properties of both 
data-sets are briefly outlined below. 

2.1 The Subaru/XMM-Newton Deep Survey 

The Subaru/XMM-Newton Deep survey (SXDS) primarily consists 
of optical and X-ray imaging covering an area of ~ 1.3 square de- 
grees, centred on RA=02: 18:00, Dec=-05:00:00 (J2000). In addi- 
tion, the SXDS field has IRAC and MIPS data from the Spitzer 
SWIRE survey (Lonsdale et al. 2003), deep 1.4 GHz radio ob- 
servations from the VLA (Simpson et al. 2006a) and 850-micron 
sub-mm observations from SCUBA (Mortier et al. 2005). Of pri- 
mary concern for this study is the deep optical imaging of the field 
undertaken with Suprime-Cam (Miyazaki et al. 2002) on Subaru. 
The optical imaging consists of 5 over-lapping Suprime-Cam point- 
ings, and covers an area of ~ 1.3 square degrees. The whole field 
has been imaged in the BVRi'z' filters, to typical 5a depths of 
B = 27.5, V = 26.7, R = 27.0, i! = 26.8 and z = 25.9 
(2"— diameter apertures). 

2.2 The UKIDSS Ultra Deep Survey 

The Ultra Deep Survey (UDS) is one of five near-infrared surveys 
being undertaken with WFCAM (Casali et al. 2006, in preparation) 
on UKIRT, which together comprise the UKIRT Infrared Deep Sky 
Survey (UKIDSS). The UDS will take seven years to complete, and 
will provide ultra-deep JHK imaging of a 0.8 square degree area 
(centred on the SXDS field), to 5a point-source limits of J = 26.0, 
H = 25.4 and K = 25.0. The UDS early-data release consists of 
the first twelve hours of JK imaging data and reaches 5a limits of 
J = 22.5 and K = 22.5 within a 2"— diameter aperture. For this 
paper we have used an improved version of stacking and catalogue 
extraction constructed by the current authors, rather than the default 
version in the WFCAM Science Archive. This will be described in 
Foucaud et al (2006). The authors of this paper are also members 
of the UKIDSS consortium, so we intend that our improved stack- 
ing and source extraction will be made available to help to develop 
the final WSA database. Due to a small change in the UDS field 
centre shortly after the beginning of the survey, the current JK 
imaging is not uniform over the entire 0.8 square degree field. Con- 
sequently, for the purposes of this paper, we restrict our analysis to 
the 0.6 square degree central region which has uniform JK data 
and optical Subaru coverage. 



3 HIGH-REDSHIFT CANDIDATE SELECTION 

Our sample of bright LBGs at z > 5 was initially selected from 
the Subaru optical imaging, followed by SED fitting of the Sub- 
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Table 1. Names, optical positions and 2"— diameter aperture photometry for the final nine high-redshift LBG candidates (quoted limits are lcr). The consistent 
image quality of the optical+near-infrared data (0.7"<FWHM<0.8") means that differential aperture corrections between bands are small (< 0.1 magni- 
tudes). All aperture magnitudes have been confirmed manually to check for errors in the original catalogue photometry. Four objects (MCD2, MCD5, MCD8 
& MCD9) have photometry calculated in a 1.6"— diameter aperture to avoid possible contamination from nearby sources (as a result we note that formally 
MCD9 has a colour of R — z' = 2.90 ± 0.27, rather than R — z' > 3.0). Although the majority of the aperture photometry in the i' and z' filters has S/N 
> 10, a minimum error of 0.1 magnitudes was adopted during the SED fitting process to account for possible zero-point errors. 



Source 


RA(J2000) 


DEC(J2000) 


B 




V 


R 


i' 


z' 


J 


K 


MCD1 


02:16:27.81 


-04:55:34.1 


> 29.25 


> 


28.40 


> 28.70 


27.33 ±0.31 


24.63 ±0.07 


> 24.28 


24.22 ± 0.73 


MCD2 


02:16:57.29 


-04:52:00.8 


> 29.49 


> 


28.64 


28.92 ±0.74 


26.31 ±0.11 


25.04 ±0.08 


24.20 ±0.60 


23.99 ± 0.52 


MCD3 


02:17:02.70 


-04:56:59.3 


> 29.25 


> 


28.40 


> 28.70 


26.15 ±0.11 


24.63 ±0.07 


23.71 ± 0.51 


23.37 ±0.39 


MCD4 


02:17:02.90 


-05:23:17.3 


> 29.25 


> 


28.40 


> 28.70 


26.09 ±0.10 


24.63 ±0.07 


> 24.28 


23.68 ± 0.50 


MCD5 


02:18:26.56 


-05:01:51.0 


> 29.49 


> 


28.64 


> 28.94 


26.35 ±0.11 


25.00 ±0.07 


24.15 ± 0.58 


23.53 ± 0.37 


MCD6 


02:18:47.30 


-05:21:04.9 


> 29.25 


> 


28.40 


28.38 ±0.60 


25.40 ± 0.06 


24.25 ±0.05 


23.89 ± 0.57 


23.99 ± 0.62 


MCD7 


02:18:53.20 


-04:40:40.5 


> 29.25 


> 


28.40 


> 28.70 


25.96 ±0.10 


24.64 ±0.07 


24.28 ± 0.75 


23.17 ±0.34 


MCD8 


02:19:00.46 


-04:44:58.7 


> 29.49 


> 


28.64 


27.54 ±0.26 


25.39 ± 0.05 


24.36 ±0.04 


> 24.52 


> 24.51 


MCD9 


02:19:04.30 


-04:47:55.2 


> 29.49 


> 


28.64 


27.52 ±0.26 


25.67 ±0.06 


24.62 ±0.05 


23.92 ± 0.49 


23.91 ± 0.49 



aru+UDS data to isolate robust candidates. The optical selection 
criteria were as follows: 

• z < 25 

• non-detection (< 2a) in the B and V— bands 

• R - z' > 3 

The magnitude cut at z < 25 was chosen to fully exploit the area 
of the UDS, and to isolate the most luminous/massive z > 5 galax- 
ies which cannot be studied using smaller-area surveys. Further- 
more, from a practical point of view, for objects with z' > 25 the 
shape of the SED long-ward of the z' — band cannot be properly 
constrained (with the early-release UDS data), making it impos- 
sible to identify robust candidates (see Section 3.2). The require- 
ments for non-detections in the B and V— bands, together with a 
red R — z' > 3 colour, are designed to ensure our selection func- 
tion has a sharp low-redshift cut-off at z = 5. At z > 5 both 
the B and V— bands sample rest-frame wavelengths short-ward of 
Lya, and any genuine high-redshift LBGs should not be detected at 
> 2a significance in these bands. Moreover, in the redshift interval 
5 < z < 6 the Subaru R and z'— filters bracket the onset of absorp- 
tion by the Lya forest, and simulations demonstrate that a colour- 
cut of R — z' > 3 effectively excludes galaxies at z < 5. Finally, it 
should be noted that during the optical selection process we did not 
actually impose the standard i— drop criterion. The reason for this 
is that the standard colour cut (i.e. isu — Z850 > 1-3; Bunker et al. 
2004) can exclude galaxies in the redshift range 5.0 < z < 5.5. In 
contrast, for the purposes of this study, we are interested in includ- 
ing all possible candidates at z > 5, which is better achieved using 
the R — z > 3 colour-cut. However, as can be seen from Table 1, 
all of the final nine LBG candidates satisfy i' — z' > 1.0, fully 
consistent with the expected colours of z > 5 LBGs in the Subaru 
filter set. 



3.1 Optical+near-infrared model fitting 

Following the application of the initial optical selection criteria, 
the next step in the selection process was to analyse each poten- 
tial candidate with our template-fitting photometric redshift code 
(Cirasuolo et al. 2006, in preparation), which is largely based on 
the public package HYPERZ (Bolzonella, Miralles & Pello 2000). 
The photometric redshift for each galaxy is computed by fitting the 
observed multi-wavelength (BVRi' z' JK) photometry with syn- 



thetic galaxy templates. We adopt the stellar population synthesis 
models of Bruzual & Chariot (2003) for the galaxy templates, as- 
suming a Salpeter initial mass function (IMF) with a lower and up- 
per mass cutoff of 0. 1 and 100 Mq respectively. Both instantaneous 
burst and exponentially declining star formation models - with e- 
folding times in the range 0.1 < r(Gyr) < 15 - were included, all 
with fixed solar metallicity. The code accounts for dust reddening 
by following the Calzetti et al. (2000) obscuration law within the 
range < Ay < 10, and accounts for Lyman series absorption 
due to the HI clouds in the inter galactic medium according to the 
Madau (1995) prescription. In addition to the best-fitting photomet- 
ric redshift, the code also returns best-fit values for: SED type, age, 
mass and reddening. 

In addition to providing redshift and stellar mass estimates, 
the principle reason for performing the optical+near-infrared SED 
fits was to clean the sample of low-redshift interlopers. A simple 
application of the optical selection criteria described above pro- 
duced an initial sample of 74 candidates. By making full use of 
the extra information provided by the near-infrared UDS data, 65 
members of this initial sample were eventually excluded as low- 
redshift contaminants. The names, positions and photometry for the 
final nine LBG candidates are listed in Table 1, and the fits to their 
optical+near-infrared photometry are shown in Fig 1. In the next 
section we discuss in detail the potential sources of contamination, 
and the strategies adopted for excluding them from the final sample. 

3.2 Sources of contamination 

In the current study there are three types of contaminants which are 
of potential concern: ultra-cool galactic stars (M, L or T dwarfs), 
extremely red objects (EROs) at z ~ 1 and high-redshift quasars. 
It is important to develop strategies for excluding these contam- 
inants based on their optical+near-infrared SEDs because, unlike 
previous HST studies, ground-based resolution makes it impossi- 
ble to exclude stellar and QSO contaminants on the basis of being 
spatially unresolved. 

3.2.1 Ultra-cool galactic stars 

Due to the ~ 0.8" FWHM resolution of the available ground- 
based imaging of the SXDS/UDS field, high-redshift LBGs are ex- 
pected to be unresolved point sources. Consequently, the prospect 
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Figure 1. The best-fitting SEDs to the photometry of the final nine LBG candidates (see Table 1) in numerical order, horizontally from the top left. The 
best-fitting parameters are listed in Table 2. The lower panels show the value of \ 2 f° r the fits as a function of photometric redshift (marginalised over all other 
free parameters). 



of significant contamination of the final sample by ultra-cool galac- 
tic stars is potentially serious. Ultra-cool galactic stars (M, L or T 
dwarfs) perfectly mimic the redshifted SEDs of z > 5 LBGs in 
the optical (Hewett et al. 2006), and would certainly pass our initial 
optical selection criteria. Consequently, in order to exclude stel- 
lar contaminants it is necessary to rely on the fact that ultra-cool 
galactic stars exhibit redder z' — J colours than genuine z > 5 
LBGs. To quantify this we used the library of spectra maintained 
by Sandy Leggett 2 to calculate the likely colours of M, L and T 
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dwarfs in the Suprime-Cam+WFCAM filter set. This process re- 
vealed that we could efficiently exclude the cooler L and T dwarfs 
from the final sample by rejecting any potential candidate which 
was detected in the UDS EDR J— band imaging at > 2a signifi- 
cance. This is because the red z' — J > 1.5 colours typical of L and 
T dwarfs means that, at z' < 25, they should be significant (> 3a) 
detections in the EDR J— band imaging. However, unfortunately 
the potential for contamination by M dwarfs is much more signifi- 
cant. Due to the fact that none of the final nine LBG candidates are 
formally detected in the J— band (all < 2a detections), the limits 
on their z' — J colours are typically z' — J < 1.0, compatible with 
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Figure 2. BVRi' z'JK and 3.6/rni postage stamp images of the stacked data for all nine of the final high-redshift LBG candidates. Each stamp is twenty arcsec 
square (except the 3.6/rni stamp which is ten arcsec square), and is displayed using a grey-scale which is a linear stretch ±3cr around the mean sky level. The 
name of the filter is shown in the bottom-left corner of each stamp, and the position of the stacked LBG candidate is highlighted using an arrow. 



those of M dwarfs with i' — z' > 1.0. To tackle this issue, as part 
of the SED fitting we used the available M dwarf spectra (spectral 
types Ml to M9) as templates to fit the LBG candidates' photom- 
etry. For seven of the nine final candidates, using galaxy templates 
produced significantly better fits to the LBG photometry (Table 2) 
than could be achieved with the M dwarf templates. However, for 
two of the final candidates (MCD2 & MCD3) the M dwarf SED fits 
were of comparable quality to those achieved with z > 5 galaxy 
templates. Consequently, it remains possible that these two objects 
are M dwarf contaminants. However, we have chosen to keep them 
in the final sample because z > 5 LBG templates provide a very 
good fit to their photometry and, even including these two objects, 
a stacking analysis suggests the final candidate list of nine objects 
is not heavily contaminated by M dwarf stars. 

3.2.2 Stacking analysis 

In order to confirm that the final sample of nine LBG candidates 
is not heavily contaminated by galactic M-dwarf stars, we stacked 
(averaged) the images of the final nine candidates in all available 
bands. Postage stamp images centred on the stack of all nine high- 
redshift LBG candidates in the BVRi' z' JK and 3.6/mi Spitzer 
bands are shown in Fig 2, and the SED fit to the stacked photometry 
is shown in Fig 3. 

Several points regarding the stacked image and accompany- 
ing SED fit are noteworthy. Firstly, due to the extra >1 magnitude 
in depth achieved by stacking, it is possible to confirm that the 
average SED of the final candidates is robustly undetected in the 
bluest optical bands to 2a limits of B = 29.5 and V = 28.7. Sec- 
ondly, the colours of the LBG stack are i' - z' = 1.28 ± 0.14 
and z' - J = 0.22 ± 0.35. An M-dwarf star with i' - z' = 1.3 
is expected to have z' — J ~ 1.0, significantly redder than the 
z — J colour of the LBG stack. This alone is enough to strongly 
suggest that the final candidate list is not heavily contaminated 
by M-dwarfs, and is confirmed by the fact that the best-fit to the 
stacked BVRi' z' JK photometry using an M-dwarf template can 
be confidently excluded (Ax 2 = 25). Moreover, although each of 
the final nine candidates is individually undetected in the 3.6/im 



SWIRE data, the stack does provide a marginal ~ 1.5ct detection 
(m.3.6 = 24.0 ± 0.5), entirely consistent with a prediction based on 
the individual SED fits shown in Fig 1. Crucially, if the final can- 
didate list consisted largely of ultra-cool galactic stars, there would 
be no detection at 3.6/im. 

There is one final comparison which can be performed using 
the stacked data to provide further information on the reliability of 
our selection technique. As previously mentioned, in the GOODS 
CDFS field there is one bright z > 5 LBG known with £850 < 25 
(SBM03#03), spectroscopically confirmed to be at z — 5.78 by 
Bunker et al. (2003). This object is the only high-redshift LBG 
within the deep HST survey fields which is comparable in bright- 
ness to the candidates selected here. In the right-hand panel of Fig 3 
we show the photometry for this object, together with our SED 
fit. Reassuringly, our SED fit accurately recovers the spectroscopic 
redshift {z v hot = 5.7 ± 0.1) using best-fitting model parameters 
very similar to those required to fit the photometry of our nine high- 
redshift LBG candidates. Moreover, it can clearly be seen from 
Fig 3 that the SEDs of SBM03#03 and our LBG stack are very 
similar, providing additional evidence that our final candidate list 
is robust. 

3.2.3 Extremely red objects 

Based solely on our optical selection criteria, contamination of the 
final sample by EROs at z ~ 1 is possible if the 4000A break is 
confused with the Lyman break. However, given the extra informa- 
tion from the UDS EDR near-infrared data, we can be confident 
that ERO contamination is not an issue. As can be seen from Fig 1, 
none of the final nine candidates displays a problematic degener- 
acy between plausible solutions at low and high redshift. Although 
an alternative photometric redshift solution at z ~ 1 inevitably 
exists for each candidate, in every case the low-redshift solution 
can be excluded at ^> 99.9% confidence (Ax 2 > 18). This is con- 
firmed by our SED fit to the stacked LBG photometry (Fig 3) which 
shows the ERO solution at z ~ 1 to be completely unacceptable 
(Ax 2 = 75). This clear separation between high and low-redshift 
solutions is particularly notable given that during the SED fitting 
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Figure 3. The left-hand panel shows the best-fitting SED to the stacked photometry of the final nine high-redshift LBG candidates. Due to the extra depth 
provided by stacking, the SED of the LBG stack includes a detection at 3.6/xm from the SWIRE survey data. Furthermore, the LBG stack remains undetected 
in the B and V— bands to la limits of B = 30.25 and V = 29.45. As discussed in the text, the SED of the LBG stack is not compatible with that of a z ~ 1 
ERO or an ultra-cool galactic star, strongly suggesting that the final LBG candidate list is not heavily contaminated. The right-hand panel shows our SED fit to 
the brightest LBG known in the GOODS CDFS (SMB03#03) which has a spectroscopic redshift of z = 5.78 (Bunker et al. 2003), where the photometry has 
been taken from the GOODS MUSIC catalogue (Grazian et al. 2006). Our SED fit recovers the spectroscopic redshift Zphot = 5.7 ± 0.1, and has an age and 
stellar mass (5 X 10 10 M ) very similar to the SED fit performed by Eyles et al. (2005). The similarity between the SEDs of the LBG stack and SMB03#03 
provides further evidence of the reliability of our selection technique. 



process, extreme values of intrinsic reddening (0 < Ay < 10) 
were fully explored. 

From a more empirical perspective, we have also investi- 
gated the potential for contamination using a sample of 3715 
R-K > 5.3 (Vega) EROs with a A'-band limit of K tot < 22.2, 
selected from the same data-set as our LBG candidates (Simpson 
et al. 2006b). Within this sample there are 2140 EROs with photo- 
metric redshifts in the range 0.8 < z < 1.4 and z' < 25. As ex- 
pected, none of these EROs display a statistically acceptable pho- 
tometric redshift solution at z > 5. The fundamental reason for 
this is that none of them have the blue SED slope long-ward of the 
z' — band displayed by our final z > 5 LBG candidates. Perhaps 
the most convincing evidence for this comes from the SED of the 
stacked photometry of all nine z > 5 LBG candidates (Figs 2 & 
3), which has a colour of z' — K = 0.73 ± 0.25. For compari- 
son, the average z' — K colour of z ~ 1 EROs in the UDS field is 
z' — K — 2.32 ± 0.32 (where the quoted uncertainty is the standard 
deviation). 



3.2.4 High-redshift quasars 

High-redshift quasars in the redshift range 5.0 < z < 6.5 have pre- 
dicted i'—z' and z' — J colours which are comparable with those of 
our final nine LBG candidates (Willott et al. 2005). However, none 
of the final candidates are detected in the deep 0.2 — 12 keV band 
X-ray imaging of the SXDS with XMM-Newton. The sensitivity 
of the XMM-Newton SXDS observations varies somewhat over the 
SXDS field, but the typical X-ray flux upper limits imply X-ray lu- 
minosity limits of ~ (5 — 10) x 10 44 erg s" 1 (rest-frame energy 
band ~ 1 — 70 keV). Moreover, based on the latest constraints on 
the QSO luminosity function at z ~ 6 (Willott et al. 2005), con- 
tamination of our sample from high-redshift QSOs is expected to 



be minimal (conservatively < 1 QSO is anticipated with z < 25 
over the SXDS/UDS field). 



3.2.5 Lensing 

Finally, given that we have deliberately selected the brightest ob- 
jects of their type at z > 5, it is important to at least consider the 
possibility that some of the final LBG candidates may be lensed. 
The first scenario to consider is the prospect of moderate ampli- 
fication due to a lensing geometry where the LBG candidates lie 
outside the Einstein radii of lensing objects along the line-of-sight. 
If we model the potential lens as a single isothermal sphere, then 
the amplification is : fi = e _g E , where 9e is the Einstein radius 
9e = 4-7r(^) 2 ) av i s the velocity dispersion of the lens- 
ing galaxy, D$ is the angular diameter distance of the source and 
Dls is the angular diameter distance of the source from the lens. 
Four of the final nine LBG candidates have nearby companions 
within 2. 5" of the line-of-sight, which could potentially produce 
amplification via lensing. To investigate this we have estimated 
the velocity dispersions and consequent amplification for these ob- 
jects, using their photometric redshifts and the Mi — ov corre- 
lation at 2 ~ from Bernardi et al. (2003). This calculation re- 
veals that none of the companion objects are compatible with hav- 
ing ov > 100 km s _1 , and that any amplification from lensing is 
therefore small (i.e. fi < 1.1). 

The second scenario to consider is the prospect of high am- 
plification due to extremely close alignment along the line-of-sight 
between the LBG candidates and potential lensing objects in the 
foreground. Although difficult to rule-out completely, the remark- 
ably clean nature of the LBG candidates' optical photometry sug- 
gests that it is quite unlikely. For the z > 5 LBG candidates the 
largest cross-section for lensing occurs at z ~ 0.8. Consequently, 
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R-z 1 

Figure 4. The R—z' versus z' — J colour-colour plane. The final nine high- 
redshift LBG candidates are shown as filled circles (small off-sets have been 
applied to several of the candidates to prevent overlapping). The filled star 
shows the location of the stacked LBG photometry. The open squares are 
R — K > 5.3 (Vega) EROs selected from the UDS EDR (Simpson et al. 
2006b) which are undetected in the B and V— bands and have z'— band 
magnitudes in the same range as the LBG candidates (24 < z' < 25). The 
shaded area shows the locus of M, L and T dwarfs (light, medium and dark 
grey shading respectively) based on the data of Hawley et al. (2002) and 
West, Walkowicz & Hawley (2005). 



if we consider an L* elliptical at z = 0.8 as the potential lens, we 
can predict that it would have apparent magnitudes of K ~ 19.9, 
V ~ 25.5 and B ~ 26.6 (assuming M* K = -22.9; Pozzetti et al. 
2003). These magnitudes are completely incompatible with those 
of the stack of the final nine LBG candidates, which is undetected 
to 2a limits of B — 29.5 and V = 28.7, and has a K— band mag- 
nitude of K = 23.86 ±0.23. Consequently, the average luminosity 
of any potential lensing objects would have to be < 0.03 L* to avoid 
detection in the stacked photometry, which leads us to conclude that 
the probability of lensing is negligible. 



3.2.6 Colour-colour selection 

Finally, for completeness, in Fig 4 we show the location of the final 
nine high-redshift LBG candidates on the R — z' versus z — J 
colour-colour plane. Also shown in Fig 4 is the locus of M, L and T 
dwarfs and those EROs from Simpson et al. (2006b) which satisfy 
our selection criteria of being undetected in the B and V— bands, 
and have z — band magnitudes in the same range as the LBG can- 
didates (24 < z' < 25). Although Fig 4 suggests that traditional 
colour-colour selection based on R — z' > 3 and z' — J < 1 
would successful isolate our high-redshift LBG candidates from 
low-redshift EROs, it also illustrates the potential for contamina- 
tion by M dwarf stars (as discussed in Section 3.2.1). However, 
once again, the location of the stacked LBG photometry on the 
R — z' versus z' — J plane strongly suggests that the final LBG 
candidate list is not significantly contaminated by low-redshift in- 
terlopers. For the purposes of obtaining a robust sample of high- 
redshift LBG candidates, Fig 4 further emphasises the importance 
of modelling the full BVRi'z'JK SEDs, using both galaxy and 
stellar templates, rather than relying on simple colour-colour selec- 
tion. 



4 MODELLING RESULTS 

In Table 2 we list the best-fitting galaxy template parameters re- 
sulting from the optical+near-infrared SED fits to each of the final 
nine high-redshift LBG candidates (as illustrated in Fig 1). All of 
the candidates have best-fitting photometric redshifts in the range 
5.1 < z < 5.9, validating our original selection criteria. Moreover, 
as previously discussed, none of the candidates displays a plausi- 
ble low-redshift solution, despite the fact that an extreme range of 
intrinsic reddening (0 < Av < 10) was explored during the SED 
fitting process. This is potentially crucial because, as highlighted 
by Dunlop, Cirasuolo & McLure (2006), it is possible to confuse 
extremely dusty (Av ~ 4 — 6) low-redshift objects with genuine 
high-redshift galaxies if the range of reddening explored is con- 
strained to moderate values (e.g. Av < 2). 



4.1 Ages and reddening 

Although the age of the Universe at z ~ 5 (1.1 Gyrs) was not used 
as a prior in the model fitting, reassuringly the best-fitting ages for 
all the final nine candidates naturally satisfy this constraint, and are 
consistent with formation redshifts in the range 5.6 < z for < 9.0. 
As is expected given the LBG selection criteria, there is no evi- 
dence in our sample for substantial amounts of reddening. Indeed, 
the range of reddening displayed by our sample (0 < Av < 1) is 
perfectly consistent with that found for LBGs at z ~ 3 by Shapley 
et al. (2001). 



4.2 Stellar masses 

By concentrating exclusively on the brightest (z' < 25) candidates, 
this study was designed to investigate the most massive LBGs at 
z > 5. The success of this policy can be seen from Table 2, which 
shows that six of the nine final candidates have estimated stellar 
masses > 5 x 10 10 M , and five have estimated masses > 10 11 M©. 
If these stellar mass estimates are accurate, then the LBG candi- 
dates are among the most massive galaxies yet discovered at these 
redshifts (see discussion in Section 5), and are comparable to the 
highest stellar masses found for LBGs at z ~ 3 by Shapley et 
al. (2001), who also adopted a Salpeter IMF. In fact, these stellar 
mass estimates suggest that several of the z > 5 LBG candidates 
have already built-up a stellar mass comparable with M* tars today 
(~ lO^MosCole etal. 2001). 

However, given the current depth of the near-infrared data 
from the UDS EDR, and the fact that the SWIRE data covering 
the UDS field is not deep enough to individually detect the LBG 
candidates, it is clear that on an object-by-object basis the stellar 
mass estimates must be regarded with some caution. The first issue 
to consider is the range of stellar masses which are allowable within 
the SED templates we have adopted to fit the optical+near-infrared 
photometry. By identifying the SED templates with the lowest and 
highest masses which still provide a statistically acceptable fit to 
the observed photometry (conservatively Ax 2 < 10, marginalised 
over all other free parameters), we have calculated the allowable 
stellar mass range for each of the LBG candidates (Table 2). This 
calculation reveals that the photometry of most of the LBG candi- 
dates can be acceptably reproduced (although with a worse \ 2 ) by 
SED templates with a factor of ~ 3 less stellar mass. 

In addition, there is also the added uncertainty introduced by 
the choice of a specific IMF. For the SED fits presented in this pa- 
per we have made the standard choice of adopting a Salpeter IMF, 
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Table 2. The best-fitting parameters from the optical+near-infrared SED fitting. Columns list the name, SED template, \ 2 , primary redshift solution, la 
redshift range, extinction, age, stellar mass and allowed stellar mass range (see text for discussion). Although a wide range of different starformation histories 
were considered, in all cases the best-fitting SED template was either l=instantaneous burst or 2=exponential decay model (e-folding time r = 0.3 Gyr). The 
photometry for the two objects highlighted with a * symbol can be fitted with an M dwarf stellar template with a comparable x 2 , and these objects cannot be 
firmly ruled out as possible stellar contaminants. 



Source 


SED 


x 2 


z 


Az 


Ay 


Age/Myr 


Log(M/M ) 


AM 




MCD1 




2.6 


5.85 


5.76< 2 <6.20 


0.0 


57 


10.3 


10.3 < M < 


10.8 


MCD2 


1 * 


2.5 


5.38 


5.08< z < 5.56 


0.8 


57 


10.8 


10.3 < M < 


10.9 


MCD3 




0.4 


5.47 


5.16< z < 5.62 


1.0 


57 


11.2 


10.6 < M < 


11.3 


MCD4 




2.1 


5.29 


5.16< z < 5.54 


0.0 


181 


11.0 


10.5 < M < 


11.1 


MCD5 




2.5 


5.13 


5.00< z < 5.38 


0.0 


255 


11.1 


10.6 < M < 


11.3 


MCD6 




0.5 


5.26 


5.12< z < 5.42 


0.0 


114 


10.6 


10.3 < M < 


10.7 


MCD7 




0.4 


5.18 


5.02< z < 5.44 


0.4 


181 


11.3 


10.7 < M < 


11.4 


MCD8 




7.0 


5.31 


5.22< z < 5.38 


0.0 


57 


10.0 


9.9 < M < 


10.1 


MCD9 


2 


2.1 


5.41 


5.30< z < 5.54 


0.6 


509 


11.2 


10.6 < M < 


11.5 



for ease of comparison with previous results in the literature. How- 
ever, it is widely recognised that the Salpeter IMF results in higher 
stellar mass estimates than other popular choices. For example, re- 
fitting the LBG optical+near-infrared photometry with a Kennicutt 
or Chabrier IMF would produce stellar mass estimates a factor of 
~ 1.5 lower. In conclusion, it is clear that the individual stellar 
mass estimates for the LBG candidates are very likely uncertain to 
within a factor of ~ 5. 

In order to obtain a more robust estimate of the typical mass 
of the z > 5 LBGs, we have also calculated stellar mass estimates 
based on the stacked LBG photometry. The obvious advantage of 
this is that the extra depth provided by stacking produces much 
more robust photometry in the J and K— bands, and also provides a 
detection at 3.6/im from the stacked SWIRE data. Using a straight- 
forward average stack of the LBG data (see Figs 2 & 3) our SED 
fit returns a stellar mass estimate of ~ 5 x 10 10 Mq. However, this 
stack is biased due to the inclusion of the two LBG candidates with 
the lowest stellar mass estimates (MCD1 & MCD8), which have 
the faintest J + K photometry. This is confirmed by an SED fit 
to a median stack of the LBG photometry, which returns a stellar 
mass estimate of ~ 1 x 10 11 M Q (as anticipated from the results in 
Table 2). In conclusion, taking into account systematic differences 
due to the choice of IMF, the evidence from stacking the LBG data 
suggests a typical stellar mass of > 5 x 10 M . 



5 MASSIVE GALAXIES AT HIGH REDSHIFT 

The results presented in the previous section suggest that, despite 
the considerable uncertainties, the typical stellar mass of the z > 5 
LBG candidates is >5 x 10 10 M . In this section we investigate 
whether the existence of such massive galaxies at this early epoch is 
consistent with ACDM structure formation and current galaxy for- 
mation models. Within this section we adopt the best-fitting stellar 
mass estimates for each LBG candidate, based on a Salpeter IMF. 
However, when necessary, we adjust our estimated number densi- 
ties to account for differences due to specific choices of IMF. 



5.1 Surface density 

The first, model independent, quantity of interest to calculate is the 
surface density of z' < 25 galaxies at z > 5. However, in order to 



accurately compute the surface density, it is first necessary to ac- 
count for the inevitable incompleteness introduced during the con- 
struction of the original SExtractor (Bertin & Arnouts 1996) cata- 
logues due to object blending. To quantify this effect we performed 
simulations based on introducing 1000 fake LBG candidates at a 
time into the z'— band images, and attempting to recover them us- 
ing the same SExtractor configuration adopted for constructing the 
original 2'— band catalogues. This immediately revealed that on av- 
erage we lose ~ 10% of possible LBG candidates due to object 
blending on the z'— band images. In addition, we then proceeded 
to run SExtractor in two-image mode on the corresponding B and 
V— band data, using the z'— band as the detection image. This re- 
vealed that we lose further LBG candidates due to contamination 
of the matched apertures in the B and V— bands by nearby com- 
panions. In around ~ 5% of cases this contamination is sufficient 
to fail our selection criteria requiring < 2a detections in both the 
B and V— bands. 

Consequently, when calculating the surface density of z' < 25 
LBGs it is necessary to scale-up by a factor of ~ 1.2. Allowing 
for this correction factor, our estimate for the surface density of 
z' < 25 LBGs at z > 5 is 0.005 ± 0.002 per square arcmin. We 
note that this figure is compatible with, although obviously much 
more robust than, a prediction based on the one previously known 
z > 5 LBG with zsso < 25 in a compilation of deep HST survey 
fields; 0.004 ± 0.004 per square arcmin (Bouwens et al. 2006). We 
also note that our final list of nine z' < 25 LBGs at z > 5 is fully 
consistent with the previous findings of Shimasaku et al. (2005) in 
the SDF Within the 767 square arcmin area of the SDF, Shimasaku 
et al. did not find any possible z' < 25 LBGs, but did identify 12 
candidates in the magnitude range 25.4 < z' < 26.6. However, it 
is important to remember that the selection technique adopted by 
Shimasaku et al. was tuned to identify LBGs in the redshift inter- 
val 5.6 < z < 6.2. Only one of our final list of nine candidates 
lies within this redshift range (MCD1). Consequently, based on our 
final sample, we would predict only ~ 0.4 LBG candidates with 
z < 25 within the SDF area which would satisfy the Shimasaku et 
al. selection criteria, entirely consistent with their finding of none. 



5.2 Number densities 

The best-fitting stellar mass estimates listed in Table 2 suggest that 
five of the LBG candidates have masses ofM>10 n M Q . In order 
to convert the estimated surface density of M > 10 M LBGs at 
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Figure 5. The estimated completeness of our LBG selection criteria as a 
function of stellar mass, based on subjecting a catalogue of simulated LBG 
photometry to the same selection criteria applied to the real data. It can be 
seen that at stellar masses of ~ 10 11 M Q our sample is expected to be 
~ 40% complete. 

z > 5 into a number density, it is necessary to compute the effective 
co-moving volume of our survey (defined by the survey area and the 
redshift intervals within which each of the LBG candidates could 
have been detected; Zmin < z < z max ). Due to our adopted optical 
selection criteria, our selection function has a sharp low-redshift 
cut-off at z m i n = 5. To compute z max we have taken the SED 
fit to each candidate and redshifted it (keeping age, reddening and 
mass constant) until the candidate fades below our magnitude limit 
(z = 25). This calculation reveals that, for the five LBG candidates 
with M > 10 11 M Q , the effective volume of the survey is equivalent 
to the full co-moving volume covered by our 0.6 square degree area 
over the redshift interval 5.0 < z < 5.6 (i.e. 3.3 x 10 6 Mpc 3 ). 

5.2.1 Completeness 

Additionally, in order to calculate a meaningful estimate of the 
number density of M > 10 11 Mq LBGs in our sample, it is also nec- 
essary to estimate our completeness as a function of stellar mass. 
To investigate this we produced simulated photometry for 10,000 
LBGs with parameters randomly selected from uniform distribu- 
tions with the following ranges: 

• Stellar masses: 10 10 M Q < M< 10 12 M Q 

• Ages: 50 Myr < age < 500 Myr 

• Extinction: 0.0 < A v < 1.0 

where the adopted parameter ranges are based on the results of the 
SED fits presented in Table 2, and are also consistent (in terms of 
ages) with recent results suggesting that reionisation occurred in 
the redshift interval 6 < z < 10 (Fan et al. 2006; Page et al. 2006). 
The simulated LBG catalogue was then subjected to the same op- 
tical and near-infrared selection criteria adopted for the real data- 
set (see Section 3). In Fig 5 we show the resulting estimate for 
sample completeness as a function of stellar mass. It can be seen 
from this figure that for LBGs with stellar masses of ~ 10 11 Mq 
we are only ~ 40% complete. Consequently, to calculate our fi- 
nal estimate of the number density of M > 10 11 Mq LBGs within 
our sample we are required to correct the observed numbers by a 
factor of ~ 2.5. Taking this correction into account, our final es- 
timate for the number density of M > 10 11 M LBGs at z > 5 
is log(0/Mpc~ 3 ) = — 5.2^0^, where the quoted error includes 
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Figure 6. The filled circle at z = 5.3 is our estimate for the number density 
of galaxies with stellar masses greater than ~ 10 11 Mq from this study. 
The data-points at z < 3 (open circles) are the estimated number densities 
of galaxies with stellar masses M > 10 11 Mq from the GOODS CDFS 
K— band selected sample of Caputi et al. (2006). The curves show the 
redshift evolution of the number density of dark matter halos with masses 
M > 1.5 X 10 12 M . The solid, dashed, dot-dash and dotted curves corre- 
spond to four different values of erg (0.90, 0.85, 0.80 & 0.75 respectively). 



a 20% contribution from cosmic variance (Somerville et al. 2004), 
but does not include possible systematics in the adopted complete- 
ness correction. If, in reality, LBG candidate MCD3 is an M dwarf 
contaminant, then the estimate number density would be ~ 20% 
lower. 



5.3 Comparison with dark matter halo number densities 

The most fundamental test which we can perform is to ask whether 
ACDM produces the required number of dark matter halos to host 
galaxies with stellar masses of ~ 10 11 M at this early epoch. 
In order to explore this issue we have calculated the number den- 
sity of dark matter halos as a function of redshift predicted by the 
Press-Schechter formalism (Bond et al. 1991) . In Fig 6 we show 
the redshift evolution of the dark matter halo number density for 
halos more massive than 1.5 x 10 12 M Q , which corresponds to 
a stellar to dark matter ratio of 15. Although the typical stellar 
to dark matter ratio of massive galaxies as a function of redshift 
is not accurately known, our choice of 15 as a representative fig- 
ure is motivated by several lines of reasoning. Firstly, assuming a 
halo occupation number of unity, the predicted number density of 
Mdm > 1.5 x 10 12 Mq dark matter halos at z = is in good 
agreement with the locally observed number density of galaxies 
with stellar masses > 10 11 Mq (Salpeter IMF; Cole et al. 2001). 
Moreover, a ratio between halo mass and stellar mass of ~ 15 at 
low-redshift is in good agreement with recent galaxy-galaxy lens- 
ing results (Mandelbaum et al. 2006). Finally, at high redshift, a 
stellar to dark matter ratio of ~ 15 is also in agreement with the pre- 
dictions of recent galaxy formation models (R. Bower, private com- 
munication). At high redshift the number density of dark matter 
halos is strongly dependent on the value of the as parameter. Con- 
sequently, in Fig 6 we show the dark matter halo number density 
evolution for four different values of a 8 (0.90, 0.85, 0.80 & 0.75). 



3 using code kindly provided by Will Percival 
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Figure 7. The filled circle is our estimate of the number density of massive 
galaxies at z = 5.3 (see text for discussion). The lower solid line is the 
number density of galaxies as a function of stellar mass at z = 5.3 from the 
latest version of the Durham semi-analytic galaxy formation model, which 
incorporates AGN feedback (Bower et al. 2006). The dotted curve is the 
latest prediction of the galaxy formation model described in Granato et al. 
(2004). The upper smooth curve shows the number density of galaxies as a 
function of stellar mass at z ~ (Kennicutt IMF; Cole et al. 2001). 



The final value of as = 0.75 is in agreement with the WMAP three 
year results (Spergel et al. 2006). 

In Fig 6 we also plot our estimated number density of LBGs 
with stellar masses > 10 11 Mq from this study (median z = 5.3). 
It can be seen from Fig 6 that, assuming a stellar to dark matter 
ratio of 15, ACDM does produce sufficient halos at high redshift 
to host our LBG candidates provided a$ > 0.8. In fact, given the 
large uncertainties associated with our estimated number density, 
our results are also fully consistent with the latest determination 
of as = 0.75. Consequently, we can conclude that our estimated 
number density of massive galaxies at z > 5 can certainly be ac- 
commodated within current ACDM models of structure formation. 

Furthermore, it has recently been suggested that ~ 60% of 
the stellar mass at z ~ 5 is missed by traditional drop-out selection 
techniques (Stark et al. 2006), which exclude high-redshift galaxies 
which are too red in the rest-frame optical to satisfy LBG selection 
criteria (see discussion in Section 6). However, even if we increase 
our estimated number density of massive galaxies at z ~ 5 by a 
factor of two, and assume erg = 0.75, ACDM can still produce the 
required number of dark matter halos if the stellar to dark matter 
ratio is ~ 10. 

In Fig 6 we also show the estimated number densities of 

> 10 11 M Q galaxies at z < 3 derived from a K— band selected 
sample of galaxies in the GOODS CDFS (Caputi et al. 2006). The 
Caputi et al. sample has a magnitude limit of K < 23.3, and is 

> 85% complete for stellar masses > 10 11 M (Salpeter IMF) at 
redshifts z < 3. As can be seen from Fig 6, the conclusions of Ca- 
puti et al. were that ~ 20% of the local mass density comprised 
by galaxies with stellar masses > 10 11 M Q was already in place 
by 2 ~ 2, and that the majority was in place by z ~ 1. Simi- 
lar conclusions have been reached by numerous studies of deep, 
small area, near-infrared surveys (e.g. Drory et al. 2005; Fontana 
2004). The results presented here strongly suggest that the increase 
in number density of massive galaxies within the redshift interval 
2.5 < z < 5.5 closely traces the build-up of suitable dark matter 
halos and that, as a consequence, only < 1% of the local density of 

> 10 11 M Q galaxies was in place by z ~ 5. 



5.4 Comparison with galaxy formation models 

It has recently become apparent that the number density of mas- 
sive, red galaxies observed at high redshift is much larger than 
would have been predicted by semi-analytic galaxy formation mod- 
els five years ago (e.g. Cimatti et al. 2002; Glazebrook et al. 2004). 
In essence this is simply a result of the fact that galaxy formation 
models based on hierarchical assembly naturally produce the most 
massive galaxies at late times. This apparent discrepancy between 
the models and observations has been described variously as "anti- 
hierarchical" behaviour, or cosmic downsizing. It is clear therefore 
that important constraints can be placed on the latest generation of 
galaxy formation models via comparison with the observed number 
density of massive galaxies already in place at z > 5. 

In Fig 7 we show the predicted number density of galax- 
ies as a function of stellar mass at z = 5.3 (median redshift of 
our LBG candidates) from the latest generation of Durham semi- 
analytic galaxy formation models, which incorporate AGN feed- 
back (Bower et al. 2006). Also shown in Fig 7 are the predictions 
(GL. Granato, private communication) from the latest version of 
the Granato et al. (2004) galaxy formation model, which is based 
on an anti-hierarchical baryon collapse scenario. The filled circle is 
our estimate for the number density of galaxies within a bin cen- 
tred on 10 11 Mq, based on the results of this study. When plot- 
ting the data-point in Fig 7, we have shifted the centre of the bin 
to lower mass by 0.15 dex, to account for the difference between 
the Salpeter IMF adopted in this paper, and the IMFs adopted in 
the Bower et al. and Granato et al. models (Kennicutt and Romano 
et al. 2002 respectively). As can be seen from Fig 7, our estimated 
number density is clearly in good agreement with the predictions of 
the Bower et al. and Granato et al. models. Interestingly, given that 
our number density is almost certainly a lower limit, both model 
predictions can accommodate a number density which is a factor 
of ~ 2 — 3 higher than our estimate, as required if LBGs constitute 
< 50% of the total stellar mass at z > 5 (e.g. Stark et al. 2006). 
Finally, although not included in Fig 7, we also note that our esti- 
mated number density of z < 25 LBGs at z > 5 is fully consistent 
with the hydro-dynamical simulations of Night et al. (2006). 



6 COMPARISON WITH RECENT RESULTS 

In this study we have exploited the large co-moving volume cov- 
ered by the SXDS/UDS data-set to identify a sample of bright 
(z' < 25) LBGs at z > 5. In this section we compare our results 
with those of three recent studies based on the extensive multi- 
wavelength data available in the GOODS CDFS (Grazian et al. 
2006; Stark et al. 2006; Yan et al. 2006). 

The GOODS-MUSIC sample (Grazian et al. 2006) consists 
of 2 — band and K— band selected samples in the southern GOODS 
field, and provides a photometric redshift for each object calculated 
from SED fits to the extensive, high-quality, HST+VLT+Spitzer 
imaging available in the field. The z— band catalogue is 100% com- 
plete to 2g5o = 25 and, although it only covers 142 square arcmin, 
it is obviously of some interest to compare the number of z > 5 
galaxies found in the MUSIC sample to the results presented here. 
Based on the results of this study, and correcting for the difference 
in area, the number of 2850 < 25 LBG-type galaxies within the 
GOODS-MUSIC catalogue should be consistent with 0.73 ± 0.30. 
Interestingly, the GOODS-MUSIC catalogue contains seven ob- 
jects with 2850 < 25 and a photometric redshift estimate of z > 5, 
seemingly very inconsistent with the results presented here. How- 
ever, our own SED fits to the MUSIC photometry for these seven 
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objects suggest that only two are believable high-redshift candi- 
dates. 

Three of the seven objects (MUSIC IDs 1 133, 8316 & 12966) 
have SEDs which are clearly stellar in nature, and have SExtrac- 
tor stellarity parameters of 0.98, 0.99 and 0.99 respectively. In ad- 
dition, MUSIC ID=7004 has a clear V— band detection which is 
inconsistent with the best-fitting photometric redshift of z v hot = 
6.91. Finally, MUSIC ID=10140 has a very unusual SED which 
would appear to be due to severe blending with bright nearby 
objects in the near-infrared and Spitzer bands. Of the original 
seven objects, this leaves two apparently robust z > 5 candidates 
with 2850 < 25 (MUSIC IDs 499 & 3094). One of these (MU- 
SIC ID=499) is the spectroscopically confirmed LBG SBM03#03 
(Bunker et al. 2003) and the other (MUSIC ID=3094) has been 
spectroscopically confirmed at z = 5.55 by the GOODS VLT 
spectroscopy campaign (Vanzella et al. 2006). Consequently, given 
that the GOODS MUSIC catalogue is complete to z 8S o = 25 (i.e 
does not employ any colour selection), and the much larger cosmic 
variance in the smaller GOODS field (~ 40%; Somerville et al. 
2004), we conclude that the number of robust z > 5 galaxies with 
2850 < 25 in the MUSIC catalogue is in good agreement with the 
results presented here. 

In their recent study, Stark et al. (2006) assemble a combined 
photometric and spectroscopic catalogue of objects in the GOODS- 
CDFS field to investigate the stellar mass density at z ~ 5. 
Within their sample Stark et al. have two objects with z > 5 and 
Z850 < 25, one with a spectroscopic redshift of z = 5.55 (MU- 
SIC ID=3094) and the other with a photometric redshift of z p hot = 
5.38 (also from the MUSIC catalogue). Our SED fits to these two 
objects successfully reproduce the redshift of the spectroscopically 
confirmed candidate, but are unable to produce a statistically ac- 
ceptable high-redshift fit to the candidate with z p hot = 5.38. Con- 
sequently, if we also include SBM03#03 (z apec = 5.78; Bunker et 
al. 2003), we again conclude that the GOODS-CDFS field contains 
two z > 5 objects with 2850 < 25. 

Interestingly, the z spec = 5.55 object in the Stark et al. sample 
is not a classic i— drop (igi4 — 2850 = 0.57) and would not have 
passed our strict selection criteria (estimated colour R — z' = 2.1 
in the Subaru filters). This has two consequences. Firstly, it brings 
the observed number of z > 5 LBGs in the GOODS-CDFS field 
with Z850 < 25 into good agreement with the results presented 
here, which correspond to 0.73 ± 0.30 objects per GOODS-CDFS 
area. Secondly, it provides further evidence for the conclusion of 
Stark et al. (2006) that traditional drop-out selection techniques are 
missing ~ 60% of the stellar mass at z ~ 5. Finally, Yan et al. 
(2006) have recently completed a study of the stellar mass density 
at z ~ 6 based on i— drop selection of galaxies in the north and 
south GOODS fields (combined area ~ 330 square arcmin). Within 
their sample, Yan et al. find two objects at z > 5 with zsbo < 25. 
This is in good agreement with the results presented in this paper, 
which correspond to 1.69 ± 0.56 objects per GOODS field (N+S). 

6.1 The absence of super-massive galaxies 

As a final comment we note here that we have found no evidence 
in this study for the existence of LBGs with masses in excess of 
3 x 10 11 M Q in the redshift interval 5 < z < 6. Although still 
uncertain, this non-detection is notable given that we should be ~ 
60% complete to LBG-type objects of this mass (see Fig 5) over a 
volume of ~ 5 x 10 6 Mpc 3 . Consequently, the detection of even one 
object of this mass within the UDS/SXDS field would have pointed 
to a number density of log(</>/Mpc~ 3 ) ~ —6.7. Our non-detection 



is in agreement with the recent search for > 3 x 10 11 M Q galaxies 
within the GOODS-CDFS field by Dunlop, Cirasuolo & McLure 
(2006). 



7 CONCLUSIONS 

In this paper we have presented the results of a study aimed at ex- 
ploiting the large co-moving volume of the SXDS/UDS data-set to 
identify the most massive LBGs at z > 5. Our main conclusions 
can be summarised as follows: 

(i) We have identified a robust sample of nine bright 
(z < 25) LBG candidates with photometric redshifts in the range 
5.1< z < 5.9. 

(ii) Our corresponding estimate of the surface density for 
z' < 25 LBGs at z > 5 is 0.005 ± 0.002 per square arcmin. 
This surface density estimate is in good agreement with, although 
much more robust than, a prediction based on the one previously 
known LBG candidate with zsbo < 25 in a compilation of the deep 
HST fields; 0.004±0.004 per square arcmin (Bouwens et al. 2006). 

(iii) SED fits to the stacked photometry of the final nine LBG 
candidates suggests that their typical stellar mass is > 5 x 10 10 Mq 
(Salpeter IMF), comparable with the most massive galaxies cur- 
rently known at z > 5. 

(iv) Based on our best-fitting stellar mass estimates, five of the 
nine LBG candidates have estimated stellar masses >lO n M . 
Our corresponding estimate for the number density of z > 5 LBGs 
with stellar masses > 10 11 M Q is log(0/Mpc -3 ) = -5. 2±o%. 

(v) This number density is found to be compatible with the den- 
sity of suitable dark matter halos predicted by ACDM structure for- 
mation models for plausible combinations of as and the ratio of 
stellar to dark matter. 

(vi) It is found that recent galaxy formation models can account 
for the existence of such massive galaxies at high redshift, even if 
LBGs only account for < 50% of the total stellar mass at z > 5. 

(vii) Despite the large co-moving volume surveyed, no evi- 
dence is found for the existence of LBGs with masses in excess 
of 3 x lO n M atz > 5. 
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